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1. INTRODUCTION 

The teim generic design denotes two related ideas. It suggests that design as an activity has a dis- 
tinct conceptual and cognitive realization from nonnlesign activities, and that it can be abstracted away 
from the particulars of the knowledge base of a specific task or discipline and studied in its own right. It 
has its origins in the design methodology research of the 1960s (Cross. 1986). At this time the observation 
was made that tlie various design methods, while they differed in particulars, shared a common pool of 
assumptions which conceived the design process as moving through the foUowing sequence of steps: (i) an 
exploration and decomposition of the problem (i.e. analysis); (u) an identification of the interconnections 
between the components; (iU) the solution of the subproblems in isolation; and finally, (iv) Uie combination 
(taking into account the interconnections) of the partial solutions into the problem solution (i.e. synthesis). 
On the basis of this observation many researchers concluded that "the logical natore of the act of designing 
is largely independent of the character of the thing designed" (Archer. 1969. p.76). However, they did not 
. go on to develop the concept to any significant extent 

Subsequently these assumptions were questioned by other researchers (Akin, 1979; Lawson, 1979) 
woiking in the different framework of NeweU and Simon's (1972) Information Processing Theory (IPT).» 
While ihe concern of the earlier researchers was with the development of "systematic design metiKxls" to 
help designers (often working in teams) to deal with the increasing amount and complexity of project infor- 
mation (Cross. 1986). the concern of the latter is with explicating the internal structures and procedures 
individual cogniUve systems use during design activity, with what Easunan (1969) called intuitive design. 

The study of uituiUve design, within an IPT framework, has become a dominant mod-, of research 
into design activity.2 But this research is moving in two directions which are rather dissatisfying from t!ie 
perspecrive of developing a cognitive theory of design. First, the research tends to be discipline-specific 
and even task-spe:ific (e.g. Kant and NeweU. 1984; Kant 1985; Steier and Kant 1985; Jeffries et al. 1981; 
UUman et al., 1986; Akin, 1979. 1986). Second, diere is a proliferation of disciplines and activities being 
labeled as "design." Tlius for example. Perkins (1986) labels the process of knowledge acquisition "design." 
Thomas (1978) analyses communication as a design process. TTiomas and Carroll (1979) assume that letter 
writing, naming, and scheduling are all design activities. The first of these trends flies in the face of the 
intuition lying behind the notion of generic des.gn. The second threatens to dr^ the word design of aU 
meaning. 

One reason for these trends is the nature of IPT itself. Within IFT design is problem-solving 
activity. But pro blem solving encompasses a wide range of cognitive activity; indeed, according to some 

!J^J^T"^ V™T '"^^y If T " P««nted by NeweU and Simon (1972). TTie uniiu- 

uiled reader if woll tefemd to this original woiic. 

r An!iT • ""'^ " • ^ "hat the development of effective 

^ . r^r? ' ^ * (deiigner'i) cognitive pioce,iei (BalUy et d. 1984). Second (here h the hope 

Urn U» .mdy of hum« deiignen wiU lead to iniighu into the automation of the d6sign proce„ (Kant «,d Ns^aS^ 
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theoreticians, all of symbolic cognitive activity (e.g. Newell, 1980). Intuitively one feels that there must be 
a description of design problem-solving activity which both captures the similarities in the problem-solving 
process across the various design disciplines and also recognizes the differences between design and non- 
design problem solving. This is the level which the term generic design infonnally tries to characterize. 
In the vocabulary of IPT, there must exist a design problem space — a problem space with major invariant 
characteristics across all design situations. However, as has been observed by a number of researchers (e.g. 
Greeno, 1978), the theory does not easily facilitate such classification. We see three interrelated reasons 
(or this "shcxtcoming." 

1) In some ways the vocabulary provided by EPT seems to be missing a layer. At the top level of 
the theory one can talk about Information Processing S /stems, Task Environments, and Prob- 
lem Spaces. But the next level down takes (xie directly to the implementation details of 
specific programs where one must talk about states and transformations at the level of the ele- 
mentary information processes. Differentiation of problem types is readily possible only at this 
lower level. There is a gap in the middle where (me intuitively feels there should be several 
intermediate levels of psychologically interesting concepts — such as generic desi^. 

2) The structure of the information processing system is underdeveloped. Except for the size of 
short-term memory (STM) and read/writu times, it does not impose many significant constraints 
on the problem space. Thus the problem space tends to be substantially task determined. 

3) But the notion of task environment has not been fully explored and exploited within the theory. 
While the theory does say that the task environment consists of (i) the goal or desire to solve 
the problem, (ii) the problem statement, and (iii) any other relevant external factors, the fact 
remains that historically, die goal or motivation of the problem solver has sunply been 
assumed, and the "other relevant external factors" have been effectively ignored.^ The 
f;mphasis has been on how the problem statement gets mapped onto the problem space. 

Within IPT there are two possible sources of invariants on the design problem space. One is the 
structure cf the task environment, the odier is the structure of the information processing s)'stem (IPS). 
One way of motivating a DPS is to identify task environments and information processing structures that 
are particular to design situations. This is precisely the strategy that will be pursued here. However, we 
will have little new to say about the structure of the EPS. Most cf the paper is concerned with explicating 
the structure of the design task environment and specifying its impact on the DPS. 

Organization and Overview of Paper: In this paper we would like to play out the intuition which 
says that the design problem space is an interesting and "natural" categorization of problem spaces. Our 
strategy will be to (i) characterize design as a radial category and flesh out the task environment of the 
' Thu ii undoubtedly du« to the influence of g«me-pUying problcnw on which the thcoiy grew up. 
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central or prototypical cases; (u) take the design task environment senously; (ui) explicate the impact of 
the structure of the task environment and the structure of the IPS on the problem space of subjects from 
three different design discipUnes; (iv) suggest that the features noted in these problem spaces will not all 
occur in a problem space where the task environment is vasUy different; (v) claim that these features arc 
invariants in the problem space of design situations and collectively constitute a design problem space. 
Two aspects of our strategy differentiate this woik from much of the current research in design: (i) we 
take the structure of L'le design task environment very seriously, and (2) we examine data from three 
different design disciplines. The paper concludes by drawing some lessons for tiie development of CAD 
systems, noting some methodological limitations, ar.d suggesting avenues for further research. We begin 
by characterizing design and the design task environment. 

2. CHARACTERIZING DESIGN & THE DESIGN TASK ENVIRONMENT 

In this section we would like to claim diat design is not a ubiquitous activity. We no more design 
all the time Uian we read aU the time, play chess all the time or engage in scientific research aU Uie time. 
But the characterizations of design in tfie cognitive science literature would have us believe that most of us 
do engage in design activity most of the time. We briefly review some of this literature and conclude by 
offering our own, raUier different, analysis. 

Periiaps the most encompassing characterization of design is due to Simon (1981, p.l30): 
Everyone designs who devises courses of action aimed at changing existing situations into 
preferred ones.... The intellectual activity that produces material artifacts is no different 
fundamentally from the one that prescribes remedies for a sick patient or the one tiiat 
devises a new sales plan for a company or a social welfare policy for a state. 

On this account, anyone dissatisfied witii existing states of affairs and attempting to transform tfiem into 
"preferred oms" is engaged in design activity. The domain of design would seem to be coextensive with 
the domain of piioblem solving."* 

An early attesnpt at circumscription is due to Reitman (1964). In a paper on iU-defined problems. 
Reitman (1964) suggested a categorization of problems into six types based upon Uie distribution of infor- 
mation witiiin a problem vector. A problem vector is a tuple of Uie form [A, B. =>J, where components A 
and 3 represent die start and terminal states respectively, and Uie.component => denotes some transforma- 
tion function. Reitman's Type2 problems correspond to our intuitive notion of design. Typical Type2 
problem statements are: 



* Which in tum is coexiensivc with thinking in IPT. 
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compose a fugue 
design a vehicle that flies 
write a short story 
design a building 
make a paper airplane 

While these statements encompass widely varying activities, Reitman observed !hat they constitute formally 
similar problems by virtue of the amount and distribution of information among the three components of 
the problem vector. In the case of the Type2 or design problems, the invariant characteristic is the lack of 
information. Fo** instance: 

(i) The start state A is unspecified (e.g. design a vehicle... with what? putty? cardboard? prefabri- 
cated parts from GM?). 

(ii) The goal state B is incompletely specifiied (e.g. how long should a story be? what should the 
plot be? how should it end?). 

(iii) The transformation function => is unspecified (e.g. how should the airplane be made?... by 
folding the p^r? by cutting and pasting?). 

After this seminal paper design problems became identified with ill-defined problems. 

Continuing the investigation of ill-defined problems, Simon (1973) argued that problem? in the world 
do not come prelabeled as "well-defined" or "ill-defined." Furthennore, according to Simon, "well-defined" 
and "ill-defined" are not mutually exclusive categories. They constitute a continuum. Where a given prob- 
lem falls on this continuum is a fimction of the stance the problem solvw takes to the problem. That is, 
the problem solver may ignore existing infnmation or supply missing information from long-term memory 
or external aids. The conclusion that follows from Simon's discussion is that what constitutes a design 
problem is determined by the intentions and attitudes of the problem solver. This is an interesting position 
that has found some acceptance in the literature (e.g. Thomas and Carroll, 1979). It does however, have 
the effect of once again opening up the flood gates as to what constitutes design activity. 

Each of these attempts at delimiting or characterizing design is due to cognitive science researchers. 
Designers typically offer very different definitions. A rather well-accepted one among designers is due to 
Eastman (1981, p.l3): "Design is the specification of an artifact that both achieves desired performances 
and is realizable with high degrees of confidence." This statement emphasizes that the product of design is 
an artifact specification, and that considerations of performance and rcalizability are integral to the process. 

Awhile each of these definiticms is interesting in its own right and has a role to play in our under- 
standing of design, none of them is sufficient for our purposes here. Design is too complex an activity to 
be captured in a one-line definition; particularly a one-liner that purports to specify necessary and sufficient 
conditions. As such, our characterization of design starts with the obseivation that design as a category 
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exhibit what Rosch (in Ukoff, 1987) cails "prototype effects." Furthemore, it is what Lakoff (1987) calls 
a radial category — a category in which there is a central, ide^^ or prototypical case aiid then some 
unpredictable but modvated variaUons. On this assumption, if one shows people a Ust of professions — 
eg. medicine, legal v/ork, aichitecuire, teaching, engineering, and rtssearch — and asks them which arc the 
best examples of design professions, they wiU aU invariably and consistendy pick out the same few cases. 
In this list we beUeve the "best examples" would be architecture and engineering. We propose to call these 
-good- or "central- or -prototypical" examples of design professions. 

Having made this observation, we propose to take a serious look at the task environment of these 
prototypical design professions. In so doing we will be using the teim "task envimnment" much more 
broadly than it is generally construed in IPT. We want to use it to encompass much of what is relevant 
bJid external the problem space and the infoimaUon processing system. The danger with Uiis move is 
tiiat cither it results in a theoretically uninteresting term — because in some sense everything is relevant ~ 
or one is obUged to say what matters and what doesn't We go tJie latter route and attempt to spccufy 
some of the more important aspects of the design task environment 



Fig. 1 approx here 



The structure of the design task environment as we construe it is depicted in Fig. 1. As a first 
approximation one can note the following overt featurcs:^ 

1) niere are many degrees of freedom in the problem statement. (This is just a positive refonnu- 
laUon of Reitman's (1964) earUer point about a lack of infoimation in design problem state- 
ments.) 

2) There is delayed/limited feedback (on die order of many hours to many months) from the 
world during problem solving. 

3) The input to the design process substantially (though not completely) consists of goals and 
intentions. The output is a specification of an artifact. 

4) The artifact must function independenUy of the designer. 

5) There is a temporal separation between tiie specification and deUvery of die artifact (wiUi 
specification preceding delivery). 

6) There arc costs associated with each and every action in the world, (i.e. There are penalties 
for being wrong 



^ No attempt ii bring m«de to be ixiuuitive. 
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7) There are no right or wrong answers, only bettei' anc^ worse ones. 

8) The problems tend to be large anc. coropHex. 

We claim that the^e are significant invariants in the task envininments of prototypical design situations, and 
we can use them as a template to identify other cases of design. To the extent that the task environment of 
a given problem situation meets or conforms to tliis template, that problem situation is a good or prototypi- 
cal example of a design situation. To the extent that a task environment varies from this template — by 
omission of one or more of the requirements — to that extent it is a less central case of design activity. 

Some problem-solving situations which fit well into tiie scheme are instructional design, interior 
design, "text book cases" of software design, and music c(xnposition. Some tasks that deviate slightly are 
writing aiul paintings Here there is usually no separation between design and delivery. The problem solver 
actually constructs tlte aitifact rather than specifying it Some activities that deviate more radically are 
classrocMn teaching, spontaneous conversation, and game playing. 

Note that we are not stipulating what is and is not a design activity. To do this we would have to 
insist that the eight task environment characteristics, or some subset of them, constitute necessary and 
sufficient conditions for design activity. We make no such claim. Rather, all we are suggesting is that we 
have here a template of some salient characteristics common to the task environment of problem situations 
that are c(msistently recognized by people as good examples of design activity. Problem situations in 
which the task environment fails to conform to this template on one or more accounts are deviations from 
the central case. In this paper we ar». only interested in central cases and thus have no interest in saying 
how fat one can deviate from the prototype and still be "really" designing. Thus, we will use the label 
'design' u> refer to situations that closely confmm to the prototypical ch- central cases. 

There are two reasons why the above might be a reasonable characterization of design for our pur- 
poses. First, it is descriptive. We look at the task environment of some designers and try to take it seri- 
ously, Tne task environment of an activity is usually overtly visible wi'Ji minimal theoretical commitments 
(though it does require some unmersion in the activity and the ability to specify the more relevant factors). 
Second, in IPT the structure of the information processing system is relatively underdeveloped, leaving the 
task environi ient as the major tool/resource for suiictaring the problem space. Furthermore, the theory 
asserts that people "are severely stimulus-bound" (Hayes and Simon, 1974, p.l97) with respect to represen- 
tation and construct a naive/transparent model of the problem based upon "the surface features of the exter- 
nal environment..." (Newell, 1980, p.714). Thus given the accessibility and the importance of the task 
environment to IPT, it seems like a good basis for classification. In the next section we examine each of 
the invariant features of the design task environment and hypothesize about their impact on the DPS. 
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3. A CASE FOR GENERIC DESIGN: THE DESIGN PROBLEM SPACE 

In the previous section we identified eight interesting invariants in the structure of the design task 
environment. These invariants are external features of design activity that '.lave been noted by various 
researchers at various times and places in the design methodology literature. But we are unaware of any 
studies in the IPT literature in which these factors are taken seriously and their cognitive implications 
sketched out. We undertake this task in this section. 

Our strategy is to examine a number of designers at work and (i) reconstruct their problem space; (ii) 
make an "explanatory connection" between the features evident in their problem spaces and the above 
noted invariants of the design task environment (DTE); and (iii) make the standard argument that the prob- 
lem space is as it is because of the structure of the DTE and the structure of the EPS. This last point 
implies that, taking the structure of the IPS as a constant, the features noted in the problem spaces of these 
tasks will not all occur in a problem space where the task environment is vastly different This leads to the 
claim that these features are invariants in the problem spaces of design situations, and collectively consti- 
tute a Design Problem Space. We are actually able to identify eight interesting invariants in the problem 
spaces of three different design disciplines. To anticipate and overview, we will claim it"i following: 

A) The many degrees of freedom in design problem statements entail extensive problem structur- 
ing.^ (section 3.1) 

B) The delayed/limited feedback from the environment, coupled with the cost of action, and the 
independent functioning requirement on the artifact entails extensive perfoimance modeling of 
the artifact in the problem space. This modeling is made possible by the fact that there is a 
temporal separation of specification and delivery phases, (section 3.2) 

C) The fiact that there are no right or wrong answers to design problems entails the use of person- 
alized evaluation functions and stopping rules, (section 3.3) 

D) The requirements of extensive performance modeling, along with the constraints of sequential 
processing and short-term memory (STM) capacity entail a limited commitment mode control 
stiategy with nested evaluation loops. This strategy is enabled by the temporal separation of 
specification and delivery, (section 3.4) 

E) ThQ neoessir/ of having to specify an artifact means that designer's must make and propagate 
commiunents. There is a tension between the limited commitment mode control strategy and 
Uie need to make commitments, (section 3.5) 

F) The size and complexity of design problems combined with the Umited capacity of STM 
require solution decomposition. However, tiie decomposition is not complete. The modules 

• By the uie of the tenni entaU and nectssitate, we are throughout making contingent causal clauns, not logical 
necesiary claims. 
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are "leaky." (section 3.6) 

G) A phenomenon closely r ' ' ' ution decomposition is the mediation of goal and artifact 
by abstraction hierarchies. It is entailed by the complexity of the problem, STM capacity^ and 
the fact that the input to tne design process substantially consists of goal statements while the 
output is an artifact specification. It is also related to the phenomenon of 
persoifu^Iized^nstitutionalized stopping rules and the making and propagating jt commitments, 
(section 3 7) 

H) The last problem space invariant we note and discuss is the use of artificial symbol systems. It 
is eniailed by the limitations on the expressive power of the "language of thought," STM ce- 
city, sequential processing, and problem complexity. It is related to and has consequences for 
the phenomenon of solution decomposition, abstraction hierarchies, the making and propagating 
of commitments, and performance modeling, (section 3.8) 

All these invariants, their interconnections, and their connections to the invariants of the DTE and the 
information processing system are explicated in the diagram in Fig. 2. While no claim of completeness is 
made for this list, it is our conti ition that collectively these invariants differentiate design problem spaces 
£tom non-design problem spaces. But before actually presenting and discussing each one, a word about 
methodology is in (»-der. 



Fig 2 approx here 



, Method: The method of investigation adapted here is that oi protocol analysis (Ericson and Simon, 
1984). The data base consists of 12 protocols from 3 different design disciplines — architecture, mechani- 
cal engineering and instructional design. To illustrate and substantiate our claims for the purpose ot this 
paper we will draw upon one protocol from each of the three disciplines. The decision as to which three 
of the protocols to use was made as follows: In the case of mechanical engineering, there was only one 
protocol. There were multiple protocols for instructional design and architecture. The decision among 
them was made on the basis of the compleieness of the artifact specification and the fluency of the verbali- 
zation. 

Task Descriptions: The architecuue Uisk involved the design of an automated post office (where 
postal tellers are replaced by automated postal teller machines) for a site on the UC-Berkeley campus. The 
mechanical engineering task was to design the automated postal teller machines (APTM) for the post 
office. The instructional design task was utirelated. It called for the design of some stand-alone text based 
instruction to prepare the secretaries of a medium-sized company for a transition from typewriters to the 
Viewpoint'^ computer environment. In each case, the subjects were given a design brief which stated the 

^ Viewpoint ii an icon-bued computer environment for Xerox Stari. It suppont such functions ai electronic mail, 
filing, word proceiiini and gnphia. q 
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client's requirements and encouraged to probe the experimenter for further information and clarification. 
They were asked to "talk aloud" as they proceeded with the task, ihe sessions were taped on a video 
recorder. 

Each of the tasks arc complex, real world problems requiring on the order of weeks to months for a 
complete specification of the artifacts. We asked the architecture and mechanical engineering subjects to 
restrict their sessions to approximately 2 hours and gave the instructional designers approximately 3 hours. 
As a result, we received solutions fpccified to an incomplete level of detail. 

Subjects: Each of the 3 subjects volunteered to participate in the study. The architect (Subject-A) is 
a Ph.D. student in the Department of Architecture at UC-Berkeley. He has liad six years of professional 
experience. The mechanical engineer (Subject-M) is a Ph.D. student in the Department of Mechanical 
Engineering at Stanford University. His professional experience is more limited, but it has included the 
design of automated bank teller machines. The instructional designer (Subject-I) is a professional with 
over 10 years experience in designing industrial training material. 

Coding Procedure: The analysis of the protocols to date has been qualitative and descriptive. We 
are still in the process of identifying the nuijor components of the design problem space and arranging 
them in an explanatory fashion so as to build a model of the design process. We are not at a stage where 
we can engage in any quantitative or predictive analysis. But on the other hand, we are not limited to not- 
ing and relating everything we sec. We have a rather explicit and constrained agenda: We want to know 
how the identified ispects of the DTE impact the DPS. 

3.1* Extensive Problem Structuring 

As noted earlier, many degrees of fiecdom exist in a design problem statement (or to put it in 
P' -an's terms, there is a lack of information). This lack of information impedes the creation of a prob- 
lem space. Problem structuring is the process of finding the missing information and using it to construct 
the problem space (Simon, 1973a). It is the first step in any design activity. Large p»-ojects may require an 
alternation between problem-structuring and problem-solving phases. While some structuring is required in 
all problem situations, one of the halbnarks of design problems is tfiat tfiey require extensive struchiring. 
The extent to which problem structuring is necessary and successful determines the nature and extent of the 
problem solving that will occur. 

Each of Uie subjects in our experiment began by articulating and fleshing out their respective prob- 
lem statements. This process proceeded tiirough the follwoing steps: (1) gathering information from Uie 
design brief; (2) soliciting information and clarification from the experimenter through questions; (3) apply- 
ing knowledge of legislative constraints (e.g. building codes, in-house company standards); (4) applying 
knowledge of "technical" constraints (e.g. "laws" of structural soundness, "laws" of learning); (5) attending 
to pragmatic constraints (e.g. time, money, resources at hand); (6) bringing to bear self-imposed constraints 
or personal knowledge; and (7) negotiating constraints. WhUe each of these can bear considerable 
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discussion, only the latter tv/o will be addressed here. With rospect to (6) two questions are raised: (i) 
what is tiie form and structun^ of this personal knowledge, and (ii) how and when is it brought to bear on 
the construction of the problem space? While we liave no definitive answers to these questions, we do 
offer some preliniiiiary observations. Iei the case of (7) we illustrate the process of negotiation and com- 
ment on when and why it might occur. 

3.1.1. Vorcu and Organization of Personal Knowledge 

The personal knowledo^e our subjects used (o construct their problem spaces was organized in rich, 
intricate chunks or schemas. Two types were discernible: general schemas and domain specific schemas. 
Generally, neither surface explicitly in protocols^ but both are easily inferred firom the situation-specific 
statements that the subjects make. 

General schemas contain knowledge about Uie way(s) the world is. They are acquired over die 
course of a lifetime and are our primary means of dealing with the world. They consist of at least pro- 
cedural knowledge, abstract conceptual knowledge, and knowledge of thousands of patterns (pictorial, 
linguistic, musical, etc.). Procedural knowledge is not open to introspection (Anderson, 1982) and tiius 
does not surface in die prorocols. However, bodi die abstract conceptual knowledge and some of die pat- 
terns are visible. 

Abstract conceptual knowledge is die genetalized biowledge — principles, laws, heuristics — which 
we extract and carry away from die totality of our woridly experience. While diere is much structuie and 
coherency in die organization of diis knowle(!ge, it does not necessarily constitute a die<^. It is perhs^s 
better characterized as knowledge firagments or "knowledge in pieces'" (diSessa, 1985). It is instantiated 
and discernible in die problem space as situation-specific conceptual krowledge. For example, here is an 
excerpt firom Subjcct-A*s protocol: 

(PFl) S-A: You, after all, you probably have your parcel or your precious letter and you want to get it out, 
stamp it, or ah, have a dialogue widi a machine and see what, how much you have to pay. 
Your probably ha^e to take it out from your bag, or whatever. So you do need a sort of pro- 
tection.... I don*t want diem to get wet.... 

Underlying diis vc»:balizati(^n are two knowledge fragments at die abstract, conceptual level — beliefs 
about die use of post offices and beliefs about when and where people do and do not like to get wet 

Knowledge of patterns is knowledge diat is stored in such a direct way so diat much of die original 
pattern or form is preserved (i.e. diere is little generalization or abstractioii). This may be voluntary, such 
as when students of poetry memorize lines of text or when architecture students draw and commit to 



* Unlet I the nibject gtopt to expliL) or ritiooilliB, as one of our architecui frequently did Here u a typical excerpt 
from him: "^Now, every buUding fitting into a lite should be hannonioui with that site. Nobody argues with that The 
next thing, and compatible with the other buildings. Ah. Wn arc going from a very antisocial period, where t>uildings 
were very antisocial and withdrawn, and aggressive, and impolite, such as the one we are sUmding m, to, ah, buildings 
which are pleasant, outgoings gentle, ah« sophisticatcid and cuiuired....'' 
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memory the foims of specific buildings, or it may be involuntary, as in the case of a stimuli which the cog- 
nitive system is unable to fully comprehend and generalize. Instances of specific patterns are visible in all 
the protocols. Subject-A for instance, in atiempting to reason about an automated postal interface, immedi- 
ately retrieved and repeatedly used the "image" of an automated bank teller machine. But it was not some 
general conception of an automated bank teller but the specific Bank of America Versateller on Telegraph 
Avenue which he regularly uses. 

(PF2) S-A: I don't want tr lipve one booth after the other and having the lines, ah, like it were a Versa- 
teller, ah, kind of a service. Bank of America has that kind of approach, here on Telegraph. 
You have two, two Versatellen and usually have this long lines on the, ah, walk path. And 
who ever, ah, leaves first in one of the two, ah, then. So you have one single line for two 
machines. I am trying to avoid that... 

Domain-specific schemas arc built on top of the general schemas. They constitute the knowledge 
acquired during the years of professional training. Tney also consist of procedures, abstract conceptual 
knowledge, and patterns. Again, the procedures are not visible in the protocol. The abstract conceptual 
knowledge here seems to be less fragmentary and more "theoretical"' than in the case of the general sche- 
mas. (lliis is not surprising coasidering that it was acquired as an organized, systematic body of 
knowledge.) For example, Subject-A has an elaborate mini-theory aboi't the use and organization of 
space between buildings. His first sentences on viewing the site are; 

(PF3) S-A: Well, what comes to my mind immediately, as I told you before when I was waiting [for] you, 
I was looking at, how this is set by pathways, this, this open space in between the sports court 
yard and these diree buildings. And in thinking about die missed opportunity that people had 
here, of having a sort of more relaxed plaza, instead of being just a cross between these two 
directions. Which makes it very efficient, ah, but for sure it didn't, ah, give any contribution 
to the urban open space.... 

Similarly, Subject-I has a mini-theory about motivating, leaching, and imparting knowledge. 

(PF4) S-I: The first thing we want to do with these people is try and sell them on a system. Any time 
you change somebody firom an old system to a new system, or from what they are doing to 
what they're goiiig to be doing, or what you're expecting them to be doing, you've got to give 
them a good positive reason. Why do I really? What's in it for me, you know.... This is posi- 
live reinforcement... 

Several of these protocol fragments (PFl, PF2, PF3) are also examples of what wc call scenario 
immersion. Scenarios are frequently occurring episodes in which designers recall and immerse themselves 
in rich intricate images from their past experience. The experience in question could have been acquired 
direcUy or vicariously through some symbolic medium (e.g. reading, watching TV). These episodes seem 
to play an absolutely crucial role in the process of generation and evaluation. For instance, the scenario in 
PFl is used to generate the functional requirement "protection from rain." In PF2, the scenario is used to 
evaluate a proposed spatial configuraUon of APTMs. We say more about scenario immersion in section 
» By 'theoreUcd' U mMnt only that it is more eUboraie, complete, con»iiient, and organized. 
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3.2. 

3«1.2. Application of Personal Knowledge 

Personal knowledge structures and procedures are stored in long-term memory (I^TM). Their index- 
ing and retrieval is not well understood. Problem structuring is the process of finding and retrieving 
''relevant" schemta and instandating them into the problem space. By instantiation is meant nothing more 
than the process by which a propositi(m of the content "All public buildings are required to have ramp 
access for the handicapped" is transformed into the proposition that "This building requires a ramp access." 
As it is construed here, problem structuring is not itself a pmblem-solving activity. But the extent to which 
it is successful does determine the amount of problem solving that will need to occur. 

Subject-I was able to find, retrieve and instantiate a single powerful schema for designing training 
programs. The template came with slots mariced for lessons, sectivus* subsections, etc. He merely had to 
fill in the blanks with the content of the particular course. He generates the required content by (i) asking 
the client (experimenter) for a list of tasks the secretary would be required to perform; (u) drawing upon 
his own personal knowledge of Viewpoint; and (iii) consulting die Viewpoint manuals. Finally, die selec- 
tion of content is guided by an idealized cognitive model (ICM) (Lakoff, 1987) of what a secretary is; for 
example: 

(PFS) S»I: All this isn't going to stay in diis create and edit documents [lesson]. This is just looking at 
what*s available, and wha^ we are going to have to do. Because within this table of contents 
weVe got related information — hardware requirements and so forth that has nothing to do 
witii die secretaries, and foundation and environment Secretaries couldn*t care less.... And the 
logoff sheet properties, I, I wouldnH even teach die secretaries. That*s none of dieir business. 
They have no need for Uiat information. That I would teach your systems admiiustrator.... 

Subject*A on die odier hand seemed to find his design problem more of a challenge and exhibited 
somewhat different behavior. His initial structuring process took twenty minutes and resembled a "brain* 
storming" session. If die protocol for diis phase is recorded as a directed graph, with die nodes forming 
individual "ideas" as diey are uttered in tempcxal sequence, and the arcs connecting related nodes, dien die 
result is a lattice structure. The density and distribution of die links suggest that there are really four 
smaller structures. First diere are some site related constraints: 

(PF6) S*A: You plotted diose trees and diat would really be a sin to touch them, I diink. At least, die ever- 
greens.... As far as seating space goes, the one just below die evergreens, I wouldn't touch all 
that comer.... 

Second there is a kernel idea:^^ 



Some of the instnicdontl design subjects actually iised the Viewpoint manual to stnicuire the task. 

The early generation and faithful development of a kernel idea is an intriguing phenomenon. It has been reported 
by several researchers* including Kant and Newell (1984) and UUman et al. (1986). Wc do not have the space to pur- 
sue it here. 



Goel & Pirolli: 14 



C?F7) S-A: And what I thought is I shouldn't necessarily think of an enclose* bailding. Cause, I am in the 
middle of an open space. It would be a contradiction to place a f rmal building there. 

Third there arc some ideas about the integration of site and structure: 

(PF8) S-A: since this is the view towards the sports field, things happen over there after 5:00 p.m. I have 
seen people plaving softball and ah, firisbee, and a lot of spectacular kind of activities. And I 
might take the opportunity of using this. So ih&t people can be out there looking at the field. 
The sunset is going to be, ah, watched. Ah, my guess is that it would be a good opportunity to 
use it And then now that I think of it, I am saying, well, I could even, ah, sort of think of 
something, some structure that might use the roof of my post office to be on a sort of more 
privileged position towards the field.... 

Lastly, there are some functional ideas about the flow of mail. 

(PF9) S-A: I have to be concerned abou>. the pick up service.... Ah, I need to be able service the 
machine from behind and to have enough space to do so.... 

Thus, he was unable to retrieve a single unified plan or schema to guide his subsequent problem solving. 
He had to start his problem solving with at least four schemas and integrate them as he proceeded. This is 
a much more challenging situation than the one encountered by Subject-I. 

Sometimes the domain-specific knowledge of the designer is insufficient to stmcture the problem. In 
such a case he first tries to use his general world knowledge; if this fails the problem may be avoided, 
abandoned or not even recognized. For example, the architect (Subject-A) had no experience in designing 
user-transaction interfaces, but he was explicitly requested to do so in the design brief. He chose to 
assume a "Versateller type intcrfiace." When pressed by the experimenter to provide further details, he gave 
the following "explanation" for avoidance: 

(PFIO) S-A: the philosophy of it is that I hate an interface which is not human.... Let's leave it open. It 
might be through a keyboard, through a menu where you have a multiple selection and you 
have a ah, sort of Versateller mode to answer.... 

3.1 J. Negotiatioii of Problem Space Boundaries 

Constraints as they occur are not always desirable. Negotiation of problem space boundaries is an 
interesting resultant phenomenon exhibited by most of our subjects. It is an attempt to shift problem space 
boundaries. Often it is done to minimize search effort by transforming the problem to fit an exLsdng plan 
or template. This seems to be the motivation behind Subject-I's attempt. Subject-I, based on past experi- 
ence, believes that training programs need some minimal instruction interaction. The instruction :\e was 
requested to design on this occasion was to be completely self-contained (i.e. no instructor interaction). He 
attempted to make the current task conform to his normal mode of operation: 

(PFU) S-I: Ok. We can't negotiate you, ah, considering bringing these people in, ah, in possibly two 
groups of five, after hours, paid overtime or something, or is this already.... 

Sometimes negotiation is also used to enlarge and complicate uie problem. Subject-A attempts to do 
this. On viewing the small triangular site Iw has been given foi the proposed post office, he is not content 
to just build a post office but wants to redesign the whole area.'^ 

" The wbject ii itanding on ■ 9th floor balcony and hai a bird'i^yc view of the site. 
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(PF12) 

S-A: So, given the fact we have that triangle [i.e. the site for the post office] over there as a li>-nit And I 
cannot exceed that I suppose? 

E: Right, that, that... 

S-A: I have to take that for granted? 

E: I, I would think so. 

S-A: That's the boundary of. You do not allow me to, to exceed in, in my are^a of intervention? 
E: No, I think you should restrict it to that 

S-A; So, I am constrained to it and there is no May ! can take a more radical attitude. Say, well, look, 
you are giving me this, but I actually, I, I'd come hxk to the client and say well look, I really think 
that ycu should restructure actually the whole space, in between the building. I'd definitely do that 
if that was the case. You come to me as a client^ and come to me wfrh a triangle alone, I will give 
you an answer back proposing the whole space. Becaiise, I, I think whole space should be con- 
structed. So, that there is an opportunity to finally to plan and that space through those, ah, this 
building, open up Anthropology and, and plan the three buildings together. So, as to really make ah, 
this ah, a more communal facility.... 

The motive here is more difficult to speculate about It could be a belief that this will result in a more 
effective artifact; a desire for a larger fee; exuberance and enthusiasm for rebuilding the world in one's 
own image; etc. 

3^. Extensive Performance Modeling 

Four important aspects of the DTE converge to necessitate extensive performance modeling of the 
artifact (in its intended environment) in the design problem space. 

1) Penalty for being wrong: It is a fact about the worid that every action occurs in real time, con- 
sumes real resources, and has real consequences. In other words, it is impossible to set the 
world back as it was before the action. At best one can only take additional action (at addi- 
tional cost) to remedy the situation, but traces of tiie original action will invariably remain. 
This is as true of bending one's little finger, uttering a sentence, walking to the grocery store, 
building a house or a freeway, or putting a man on Uie moon. The difference in each of tiiese 
cases is in tiie cost and residue — tiie penalty for error. As the penalty for error increases, we 
respond by tiiinking Uirough and anticipating as many consequences of an action as possible ~- 
before acting, 

2. Autonomy of artifact: The artifact has an independent existence from Uie designer and must 
"make it on its own." The designer cannot be there to explain its significance or perform its 
function. For example, in Uie case of die stand-alone instruction, the instructional designer will 
not be in Uie classroom to respond to difficulties and questions of comprehension. He must 
anticipate the necessary inicraction and respond to it in the structure of the artifact. Such 
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anticipatioi/prcdiction requires extensive models of the artifact interacting in its intended 
environment. 

3. Delayed/Limited feedback from world: Feedback from the environment is a major mechanism 
used by adaptive systems to enhance goal achievement in the face of variable environmental 
factors. One of the most dramatic consequences of the structure of the DTE is tliat the feed- 
back loop is delayed. The design is being developed between time t and t+1 (see Fig. 1) but 
it does not interact with the world until time t+3. But this for all practical purposes is a point 
of no return. Resources have be^tn expended and the damage has been done. The feedback 
from this point can not guide the designer in the current project but only the next "similar" 
project To guide the current problem solving the designer must simulate or generate his own 
feedback between times t and t+1. 

4. Temporal separation of specification and delivQ7: There is a linear, temporal separation 
between artifact specification and delivery. In Fig. 1 the specification is complete at time r-t-i 
and the artifact constructed in the world at time t+3. Ideally the artifact is completely specified 
before construction begins.'^ This temporal separation enables the designer to model artifact 
performance — in the problem space or some external medium — to minimize damage and the 
expenditure of more substantive resources. 

Performance modeling is necessitated by the first three aspects and enabled by the fourth. 

Modeling is both internal and external to the problem space. Some of the possibilities, and the 
sequence in which they are used, are as follows: (i) entaibnents of designer's ICMs, (ii) scenario immer- 
sion, (iii) pictorial models, (iv) mathematical models, (v) mock ups, (vi) surveys, (vi) computer simula- 
tions, etc. Our subjects did not have the time or resources to make use of all these modeling devices — 
though they all pointed out when they would normally use them. They were basically restricted to their 
problem space and paper and pencil. This meant that they could take advantage of only the first four types 
of models. We will restrict our discussion to a few comments about the first two. 

The designer's ICM of the world allows for quick and automatic inferences. We have afready 
encountered an example in PF5 where Subject-! uses his "secretary ICM" to quickly evaluate whether to 
include certain material in the lessons. Such inferences do not seem to require any effort. They fall out 
automatically from the designer's idealized cognitive model of the worid. 

Scenario immersion is a more elaborate process whweby the designer pulls out a relatively concrete 
scenario from his past experience and immerses himself in it Knowing how the scenario actually tran- 
spired, he draws upon similarities between the scenario and the cunrent situation to calculate the entail- 
ments of the current situation. It is a strategy of both the first and the last resort. For example, we saw in 

"njii if of coune noc alwtyi the caie. F»tt-tnickuig ii a case of subitantiil pinJlel procMJing. But, even here, 
there are •igniflcant lelf-conuined modules — and tnon are expensive. 



o .19 

ERIC 



Goel & Pirolli: 17 

?F2 how Subject-A evaluated a one possible spatial configuration of APTM machines by doing a mapping 

between it and a previously encountered similar situation, the consequences of which he has had first-harid 

experience. Subject-M, in determining the size and height of APTM machines wanted to do a formal study 

to see how people would use the machine. However, (perhaps knowing a formal study is not possible in 

the circumstances), he immediately and without prompting indulges in scenario immersion. 

(PF13) S-M: Ok. I think [we need...] user group studies about how.... they would do the transaction. I 
think there is something about how...they're going to use it. Maybe, most student maybe 
riding bikes sometimes. Or most people, we expect them to walk, walk in. But sometimes 
maybe students [are] kind of lazy, or maybe they ride their bike or moped.... 

While their external models varied according to task demands and their pre-existing notational systems, the 
scenario immersion strategy was common across all subjects.*'^ 

33, Personalized/Institutionalized Evaluation Functions and Stopping Rulas 

It has been noted by many people (e.g. Rittel and Webber, 1974) that there are no right or wrong 
answers in design situations, but only better and worse ones. This has two interesting consequences at the 
level of the problem space. First, it means tl'at evaluation functions are often personalized or at least insti- 
tutionalized.*^ This is quite apparent in the alt ove uses of ICMs and scenario immersion. Second, the point 
at which a design is complete is a function of cognitive and personal resources. Subject-I asked to stop 
because he was tired. Subject-M reported he could not proceed any further without doing a mock-up of 
the APTM. As we did not have the resources there for him to do so, he used this as a reason to terminate 
the session. 

3.4. Limited Commitment Mode Control Strategy with Nested Evaluation Cycles 

In section 3.2 we discussed the imp(»rtance of performance modeling. Ultimately the purpose and 
value of this is fo enable the designer to anticipate the perfcmnance of the artifact aid tiie consequences of 
releasing it in the world. Since what matters is the performance of the final, complete artifact (at time 

one possible strategy is to delay evaluation until Uie specification is complete (at the end of time 
t-¥l). Evaluation at this point would certainly yield as good a value as possible, short of direct feedback at 
time t+3. But given the time, cost, and complexity involved in the design phase itself, it is neither optimal 
or feasible. That is, quite apart from Uw time and costs involved in generating a complete design and tiien 
hiiving to scrap it and start all over again, it is a fact about adaptive systems that they require continual 
feedback when engaged in any goal seeking endevour. It is simply not possible for people to work for 

Not only doei the icenario immenion phenomenon pUy a cnicul role in perfomince modeling, it »lso tems to 
be initnunenul in genention. However, we do not dijcuii this upect of it here. 

By 'inititutionaliied' is meant accepterf by a group or organization with which the designer aifoclauss himself. 
For example, in tlie caie of Stibject-I, this means in-houie company standards and practices. In the cMt of the archa- 
teci, it might be tome "movement" such ai Bauhaus, Postmodernism, etc. 
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months on end without having any indication as to the value and status of the woric with respect to the 
goal. So not surprisingly, we found that our subjects did nor. wait until the artifact was completely 
specified to evaluate its performance. 

Since the design unlolds in a quasi-linear sequence, generally starting with a kernel idea that is 
transformed and augmented until the final form emerges, another possible strategy is to evaluate com- 
ponents of the artifact as they are being generated. This would result is a linear sequence of short 
generate-cvaluate cycles. WhUe this is cognitively a very tractable strategy it can arrest design develop- 
ment by requiring strict adherence to earlier decisions. Tnat is. a decision made at one point, while attrac- 
tive in that local context, may be inappropriate in a later, more complete context With this strategy one 
would be stuck with the earUer decision. Our subjects did not use this control strategy either. 

Instead, all our subjects used a limited commitment mode control strategy (LCMCS) which incor- 
porates the best of both worlds. !t is cognitively tractable, enhances design development, and gives good 
evaluation results. It is necessitated by the essentiaUy sequential nature of symbolic processing and made 
possible by the fact that the design phase is separate fiom, and prior to, the dcUvery phase. 

If one looks at the design process at any given time, one finds that there are at least three contexts 
that the designer needs to attend to: (i) the component of the artifact cunentfy being generated or focused 
on; (u) the complete artifact in its current state (i.e. the design so far); and (iu) the projection of the 
artifact in its complete state (i.e. the final design). The LCMCS aUows the designer to take each of these 
contexts into consideration. 

As a first option, the designer can evaluate a generated or focused component on its own and make a 
decision to accept or reject it For example, the instructional designer thought of including the component 
"start with basics and finish with more complex" in a subsection entitled "What will be Tradned." He 
rejects it even before verbalizing it. (It surfaces only when the experimenter intervenes with his question.) 

(PF14) 

S-I: Ok, we've overviewed the course now just as far as the selling features. Now we're going to do a 
litUe bit of overview of what to expect [writing: "What WiU be Trained"] Ah. now what we will 
train. Ok, and we put that over.... [writing: "Six 1-hour Sessions"]. We've going to, oh hell, that's 
bullshit 

E: What was bullshit? 

S-I: Start with basics and finish with more complex. WeU of course. What in the heli else would you be 
doing? I am riot going to step you right off the end of the Titanic and ask you to swim.... 

What matters for present purposes is that the evaluation of the component was not done in the context of 
the design but stricUy locally, on its own terms. 

Second, the designer can evaluate a generated or focused component in the current context (i.e. the 
context of the design so for). This practice results in a better evaluation function and an increase in the 
number of options. He can choose to reject or accept Uw current component or he can choose to reject or 
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modify some previous decision to make the current one acceptable. For example, at one point, Subject-I 
miakes a decision to the effect that secretaries don't need to know about "waste baskets" (an icon used to 
delete computer files). A little further down he decides that they should know how to recover deleted 
icons. Then he realizes that the only way they can do this is if they know how to use "waste baskets." At 
this point he can simply reject the later decision of teaching the secretaries about recovering deleted icons, 
but instead he decides to undo the previous decision and include a section on "waste baskets." This now 
makes it possible to stick to the second decision of teaching about the recovery of deleted icons. 

FinaUy, die designer can evaluate the generated or focused component in a later more complete con- 
text (at a later time), further increasing accuracy and options. In this situation, he can accept or reject the 
current component, as in the first case; modify some previous decision to make the current one acceptable, 
as in the second case; but in addition has the option to modify some future decision to make the current 
one acceptable. For example, Subject-A during his initial structuring phase had an idea for using the roof 
structure of the post office as a seaLng platform for viewing the sports field: 

(PFIS) S-A: I could even, ah, sort of think of something, some structure that might use the roof of my 
post office to be on a sort of on more privileged position towards the field.... 

But later when he calculated the size of the structure and realized how small it would be (i.e. reevaluated it 
in the cunent, more complete context), he abandoned the earlier idea: 

(PF16) S-A: The thought that I had before, that I might use, the envelope itself, the form, the roof, ah, 
the walls, to, to implement some sort of, ah, landscape element, so as to have a major view 
towards the sports field That I am denying now..!. I really am coming back to this and see- 
ing that, after all, I won't have huge lines. After all I just have 3 booths and a loof. That's 
what I really have here. So, I'm sort of seeing the extent, ah, to which this problem will be 
heading to. 

3 J. Making and Propagating Commitments 

i design task is not complete until the artifact is completely specified. A specification is a complete, 
procedural, and declarative description, which when executed by an exteinal agent results in the construc- 
tion of the artifact. It is not sufficient to wave one's hands and talk about the artifact in some general 
terms. One must actually make, record, and propagate decisions as one proceeds, otherwise one will have 
nothing to show at llie end of the session. Each of our subjects did explicitly record and propagate their 
decisions. 

An interesting tension exists between die LCMCS and Uie need to make commitments — between 
not acting and acting rashly; between being Hamlet and being Laertes. Designers are adept at negotiating 
this tension between keeping options open tat as long as possible and making commitments. 
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3.(S* Solution Decomposition into Leaky Modules 

A major cogniUve strategy for dealing with large complex problems is through decomposition.. 
Decomposition was a major step in the normaUve models of the design methodology movement (e.g. Alex- 
ander, 1964). It has since been questioned and discredited as overly simplistic and even harmful to the 
design process. As Alexander (1965) subsequently noted, "A city is not a tree; it is a semi-lattice." Or in 
Simon's (1962, 1973b, 1977) vocabulary, the world is only nearly decomposable. But what is to be made 
of the nearly? Some interpret it to mean that one can not talk about solution decomposition in any 
significant sense. Others assume it can be ignored and continue to do clean, tree-like decompositions (e.g. 
Brown and Chandrasekaran, 1985). 

Our data shows extensive decomposition. Each of our subjects quickly and automatically decom- 
posed their problem and developed their solution in a dozen or so modules. Subject-M's modules were 
things like key pad, screen, stamp dispensary, parcel depository, and weighing mechanism. The decompo- 
sitions were discipline specific. They were not invented anew for Uie problem but seemed to be part of the 
designers' training and practices. However, equally important, the subjects did not treat the modules as 
strictly encapsulated but rather as leaky modules, A decision made in one module could have conse- 
quences in several others. The subjects seemed to have some sort of ongoing monitoring process that 
looked for interconnections across modules. 

The subjects dealt with Uie problem of "leaks" in one of two ways. One method was to plug the 
leaks by making functional level assumptions about the interconnecting modules (see section 3.7). This 
method enabled them to bring closure or encapsulation to a module and make it cognitively tractable. For 
instance, in designing the first lesson, Subject-I did not have to attend to the details of the tiiird lesson. It 
was sufficient to make some high-level functional assumptions about it. Similarly, in considering die 
height and angle of the APTM key pad, Subject-M did not attend to the detaUs of the stamp dispensary. A 
second method of deaUng with "leaks" was to engage in opportunistic behavior — to actually put the 
current module on "hold" and to attend to some of Uie interconnecting modules right Uiere and then. 

3.7. Abstraction Hierarchies Mediate Transformation of Goals to Artifact 

The input to the design process is generally a set of goals or intentions. The output of die pit)cess is 
generally a specification of an artifact. The goals come substantiaUy from the client (Uiough arc elaborated 
in discussion wiUi die designer) and are a statement of die behavior he wants die artifact to support. The 
artifact specifications arc substantiaUy generated by die designer (tfiough die cUent's brief may provide 
some guidelines at die level of die artifact) and specify diose aspects of die artifact diat he considers to be 
causally relevant in die given circumstances. Conceptually or logically, it is tempting to say tiiat die 
transformation from goals to artifact specifications is mediated by functional specifications (sec Fig. 3). On 
diis account one gets a story whereby die intentions arc carried out by means of die functioning of die 
artifact, and die function is carried out by means of die causal structure of die artifact. Bodi function and 
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causal structure have to fit the intentions, but they are only constrained, not determined, by them. In fact 
the intentions constrain (underdetermine) function, and function constrains (underdetermines) causal struc- 
ture (see Fig. 3). 



Fig 3 approx hae 

Such explicit mediation is sometimes visible in our data. For example, Subject-M, when determining 
the components and configuration of the APTM, began with a very functional vocabulary: 

(PF17) S-M: I think that functioning-wise we have some criteria. Ah, it's supposed to fulfill the require- 
ment of user to purchase the stamps, mail the letters, and weigh parcels and mail it. Cer- 
tainly there also will be register, should be something tiiat can do tiie function for registmng 
letters. And ah, certainly we expect it to be user-friendly and without requiring any training, 
and tiansparent to user.... 

At Uiis point there is no indication of how these functions will be realized. A few minutes later they are 
mapped onto device components on a one-to-one basis: 

(PF18) S-M: So I would assume tiiere is input and output devices...and we got to also have 
depository...for letters and parcels, and something for...delivering device for stamps.... And 
we also need some device to weight... 

But generally, the story that emerges from the data is not quite so clean and is closely connected to 
the near-decomposability phenomenon of the previous section. The functional specifications and the causal 
structure specifications are not two distinct ontological categories but the same category und^ different 
descriptions. Functional specifications treat the artifact, or some component of it, as a black box and attend 
only to the input and output They basically answer the question "what function will Uiis artifact, or Uiis 
part of it, accomplish?" Artifact specifications detail the causally efficacious sOiicture of the artifact. They 
answer Uie question "hov/ is the function to be accomplished?" For example, during the course of designing 
the first lesson in the training package, Subject-I worked with several different modules, interconnected in 
various ways. Some of these modules were: lessons, sections, subsections, paragraphs, sentences, and the 
choice and arrangement of lexical and grammatical elements. This COTesponds to what we called a solution 
decomposition in the previous sub-section. In addressing each of these modules die designer may choose 
to do it at various levels of abstraction or detail. The functional-causal structure distinction is just a special 
case of this abstraction process. 

The status of any module vis-k-vis die functional-causal structure distinction depends on whether a 
what or how question is asked of the module. For example: 
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What is the function of this lesson? 
How is it going to achieve this function? 
(By means of these sections.) 
What is the function of these sections? 
How are they going to achieve their function? 
(By means of these subsections.) 
What is the function of these subsections? 
How are they going to achieve their function? 
(By means of these paragraphs.) 
• etc. 

In asking the different questions, the designer is choosing to attend to different levels of detail. Ultimately, 
this regress must bottom out at a level where the artifact is completely specified. There are some interest- 
ing observations to be nuide as to where it bottoms out and the number of levels a designer explicitly con- 
siders. 

Our data indicates that the numbo- of levels explicitly attended to by a designer is a function of his 
experience and familiarity with the task, availability of relevant knowledge, and personal preferences. The 
more routine a task is, the more quickly and directly the designer can get to the low-level details, if he so 
chooses. He knows by expr,rience what type of artifiEu:t supports what type of goals and does not have to 
reason through it via "first principles." Of our three subjects, Subject-I found the task quite routine and 
traversed the abstraction hierarchy quite quickly. Subject-M, as noted earlier (PF17 & PF18), did cascade 
down several levels of function-artifact specifications. Subject-A, when confronted with designing the 
automated mail-handling system for the post office, dealt with it in strictly functional terms. He simply did 
not have the knowledge to specify lower-level details. 

However, Subject-A consciously did something that was rather interesting. In determining the 
configuraaon and location of the post office building, he purposefully stayed at a highly abstract level for 
an extended period of time so as not to crystalize or commit himself too soon to low-level details: 

(PF19) S-A: I am constantly referring to that sketch by the way. As > du can see it's ah, although it's the 
lousiest of them all, it still, still something that I, I, I, and I am not willing to do any other 
sketch at the moment. Because 1. 1 am really, trying to figure it out and I am doing it at an 
absjact level. So, that...the flow is not affected by the crystalization of an idea.... 

Thus training, personal preferences, style, and a number of pragmatic factors can affect the number of 
abstraction levels that are considered and how qiHridy one ''escends the hierarchy. This point is tied to the 
personalized evaluation function and stopping .Gi observation discussed in section 3.3. Descending too 
soon or not descending^^^ at all is a common mistake of novice designers. This : Jiales to the earlier point 
about the tension between the LCMCS and the making of commitments. 

" One of our uutnictional deiigner lubjecu tuyed «t • veiy high abitract level and refused to come down. The 
lesult wu that he haA no aitifaci <pecificationi to show at the end of the period. 
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The level of detail at which the designer chooses to bottom at depends on professional conventions 
and standards, personal preferences, style, and a host of pragmatic factors. Subject-I, for example, did not 
stop at the specification of the actual words and sentences but went on to also specify page layout and 
typeface. But he did not have to stop there either. He could also have speci^^d the chemical composition 
of the ink or the tensile strength of the paper. He chose not to. He left it as someone else's responsibility. 
He simply assumed that they would function in the "normal way" — that the ink wo'ild not dis&ilve and 
the psqper would not fall apart — and did not feel the need to provide any specifications for them. Every 
design profession has some conventions in this respect, and there is always some freedom either way that 
the designer may exercise at his discretion. 

3^. Use of Artificial Symbol Systems 

Designers often use artificial symbol systems to filter and focus infoimation and augment memory 
and pnxessing. These systems are so crucial for the problem-solving process that if they do not pre-exist 
they have to be invented before the design can proceed.*'' Their use and importance can be seen most 
dramatically in the case of architecture. It is possft)le to recognize at least seven different symbol systems 
(six of them artificial) in the architect's repertoire (see Fig. 4). Roughly they are (i) natural language, (u) 
topology ("bubble diagrams"), (iii) similarity geometry (rough sketches), (iv) Euclidean geometry (plans, 
elevations, sections), (v) affine geometry (isometrics), (vi) projective geometty (perspectives), and (vii) 
models or mockups. (Admittedly, the ^(xtespondcnce between the formal geometries and the architect's 
various drawings is only approximate, but it does serve to highlight the richness and variety of artificial 
symbol systems that are actually used.) 



Fig 4 approx here 



The symbol systems from topology to Euclidean geometry form a sort of a hierarchy. In fact, they 
map OTiio and support the abstraction hierarchy discussed in the previous section. It is possible to make 
and repre.sent distinctions at the lower levels which ^he higho- levels do not support. Similarly, it is pos.si- 
ble to make and represent distinctions at the higher, more abstract levels, which can only be made at the 
lower levels in a hidden or obscure fashion. For example, metric distinctions are preserved in Euclidean 
geomeury but not in topoiogy, and while every proposition of topology is trivially true in Euclidean 
geometry, topology does not come into its own until one abstracts away from metric and other details. 

One of our subjecu mdized that he did not have ui appropriaus symbol lystem for the development and 
tpeciflcation of the artifact and tried to develop one aa he went along. The developnisnt of symbol systons can bo seen 
on an institutional scale in the case of the emea^ing scripting and mazing systems for interactive videodiic. 
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Subject-A in his two-hour session used the symbol systems of natural language and similarity 
geometry. There are two interesting things to note in his use of these systems. (1) Moving between the 
systems automatically commits hinr. to a level of detail by selectively highlighting and hiding information. 
(2) Within a single symbol system he constructs multiple representations of the artifact. In both cases we 
want to note that these external representations are not for communicating something after the fact They 
serve an indispensable role in the generation, evaluation, and decision-making process. Once decisions are 
made, symbol systems serve to record and peipetuate them. 

As an illustration of the first point, consider the following sequence of protocol fragments and the 
accompanying diagrams in which Subjcct-A determines the form and configuration of the poss office build- 
ing: 

(PF20) S-A: But I could eventually have one single space, where all the, ah, mail is, is deliverf^l. Which 
eyentuaUy would open up in a single way and have the booths orbiting around it So that a 
given line might occur here, another one here, and another one there.... Now what I see is a 
more enclosed to itself structure. By that I want to say is that there is an inner core and 
then this roof extending around it... 

Along with this veibalization was the concunent realization of the geometric form in Fig. 5. 



Fig 5 approx here 



The relationship between the verbalization and the diagram is a one-to-many mapping. The diagram con- 
tains several elements which the verbalization does not It contains and makes very expUcit information on 
the rough size (relative to users) and shape of each unit the configuration of the units, and how the 
designer envisions the lines forming. TTiis is not an accident It is simply not possible to draw the artifact 
in sinularity or Euclidean geometry without making commitments on these issues, whetiier you are ready to 
or not" In fact a few minutes later, whUe examining Fig. 5, Subject-A expresses surprise when he real- 
izes Uie full extent of his commitment and commences to modify it. 

(PF21) S-A: I don't want to, to affect the type of line that might happen. Why did I draw this, ah, like 
something that sticks out? Ah, no. I actually want to minimize even more. So, the way I 
sec it now is I'll have to, ah, the booths [are], conceived, probably in rach a way that the 
element itself is, is really minimized as, as, ah, formal or volumetrical type of ah. interven- 
tion. We have a main structure and 1, 2, 3 interfa and the main axis.... This seems to 
work well.... 

Accompanying this verbalization is the diagram in Fig. 6. While substantially different from the diagram 
in Fig. 5, it is consistent with the original verbalization in PF20. 

In actual similarity geometty, size, of counc, is not preserved. 
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Fig 6 approx here 

Each sketch highlights information not explicit in the verbal description. As the information is expli- 
cated, it can be attended to in subsequent generate-evaluate cycles. Much of Subject-A's problem solving 
involves traversing abstraction levels via the corresponding symbol systems. Learning to traverse this 
hierarchy has some serious consequences for design development and crystalization. One must know when 
to use which system so as not to commit oneself too soon and thereby prematurely arrest design develop- 
ment At the other extreme one mu«'. learn not to stay at the higher abstract levels for an overly extended 
poriod of time and thereby jHtxluce nothing. This observation is of course related to the earlier mentioned 
tension between the LCMCS and the necessity to make commitments. 

To illustrate the second point, we note that S>ibject-A constructed four distinct representations of the 
artifact within the system of similarity geometry: si» plans, floor plans, elevations, and sections. Further- 
more, he ^nded to the various aspects of the building as they were being drawn; for example, he calcu- 
lated the vertical dimensions of the structure when drawing the elevation (see Fig. 7), not when working on 
the plan: 

(PF22) S-A: So, maybe, ah, I should go on to a section now and see how this is ah, happening, with more 
precise measures [meaning roof overhang and the glare on thj monitors].... Ah, 6 feet I 
envisioned this to be very low anyway.... Probably 2.4 meters, 2.2 meters even.... So I'd 
say that 8 feet will be the maximum height... Ah, probably we need about 2 or 3 feet to 
have all the equipment... And the lower part of the display monitor and, and keyboard will 
be perhaps 3 feet 3.5 feei^ perhaps from the ground level. 

Fig 7 approx here 



4. CONCLUSION 

This study has identified eight significant invariants in the design task environment and characterized 
their impact of the design problem space. Fig. 2 serves as a succinct summary of both our strategy and 
findings. To repeat, our major empirical findings are the following characteristics of the design problem 
space: (A) extensive problem structuring, (B) extensive performance modeling, (C) 
personalizedAinstitutionalizcd evaluation functions and stopping niles, (D) a limited commitment mode con- 
trol strategy, (E) the making and propagating of commitments, (F) solution decomposition into "leaky 
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modules," (G) the role of abstractions in the transfonnaiion of goals to artifact specificatioas, and (H) the 
use of artificial symbol systems. But in addition to noting these features, we also made "explanatory con- 
nections" between them and the invariant features of the design task environment. We make no claim for 
completeness and fully expect our characterization to grow and evolve as we examine more of our data. 
But we do expect our strategy of viewing design as a radial category, taking the design task environment 
seriously, and examining data from several design disciplines to be of continuing value in the future. At 
this stage, we cautiously suggest that while singularly these features may be found in non-design problem 
spaces, collectively they are the invariant haUmarks of the design problem space. We now conclude the 
paper by indicating some implications for CAD systems, noting some methodological shortcomings and 
suggesting directions for future research. 

4.1. Implicatioiis for Computer-Aided Design Systems 

Typically, CAD systems provide designers with a variety of tools for modeling the anticipated perfor- 
mance of an artifact during the design process. Our characterization of generic design and our empirical 
observations suggest that there are several ways in which such systems could be enhanced. 

We noted that design characteristically involves problems wiUi many degrees of freedom, requiring 
substantial collection of infonnation, problem structuring, and negotiation. Much of tiiis information comes 
from external sources or the prior experience of die designer. At first blush, hypertext tools would seem to 
be appropriate for such activities. However, as noted by Halasz (1988), making hypertext systems Uiat per- 
mit cheap input and restructuring is still a major research issue. 

Design inherenUy involves the use of design abstractions, nested generate and evaluate cycles, and a 
limitedH:ommitment mode control strategy. This suggesis tiiat designers should be able to inexpensively 
specify design absttactions and evaluate designs at any level of abstraction. The CLU language for 
software design is an attempt along tiiese lines (Liskov and Guttag, 1986). It is essentiaUy a variant of an 
object-oriented programming language tiiat allows software designers to develop procedural and data 
abstractions and specify the preconditions and entailments of U»ese abstractions without immediate concern 
for their implementation. The fact that designers appear to mix formalisms in their representations of 
artifacts suggests Uuit we have substantial woric to do in Uiis area. 

Representation is an important issue in itself. First generation CAD systems viewed die designer's 
notes and drawings only as communicative devices. Our suidies confirm die findings of Ballay et al. 
(1984) and Ullman et al. (1986) Uiat diis is simply not die case. The designer's notes and drawings play a 
crucial role in design development by selectively focusing and filtering information and augmenting 
memory and processing. This speaks for die need to develop computational environments which can sup- 
port a wide range of symbol systems. 
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Finally, we should remark on the potential role of AI in CAD systems. AI is especially appropriate 
{(X propagating the entailments of closed-world models, as is typically done in theorem-proving programs 
or problem solving programs that deal with well-structured problems. It does not fare as well in tasks with 
changing worid models; ones that are continually influenced by knowledge brought in from the external 
world or fiom past eKperience. This would seem to imply that we should not expect AI to provide highly 
automated design systems for anything but the most routine and well-structured problems that arise during 
design. However, research on hierarchical planning could provide tools for representing and evaluating 
abstract design plans. Research in knowledge acquisition tools could influence the development of CAD 
systems that acquire new design abstractions and evaluations. Research in case-based retrieval and reason- 
ing could provide tools to augment designers* use of prior knowledge in design. Intelligent advice or help 
systems that use knowledge of particular design tasks, and on-line "pattern books" might be particularly 
useful aids for novice designers or as warehouses for the design knowledge in particular disciplines or insti- 
tutions. 

4.2. Principle Shortcomings and Limitations 

As the work currently stands, there are three principle shortcomings. The first is that the whole 
analysis is based substantially on diree protocols, one each from three of the many design disciplines. In 
the short term we justify mt experiment design by noting that the methodology used is qualitative rather 
than quantitative. It does not require large numbers of subjects. As has been argued by Anzai and Simon 
(1979) there is much to be gained by the detailed analysis of a single protocol. Over the long term, we 
recognize the shortcoming and are continuing to analyze additional protocols. 

The second shortcoming is that we have not used a formal procedure tor coding the protocols. Nei- 
ther has there has been any independent coding of Ute protocols. Again, over Uie long term, we recognize 
this shortcoming as serious. In the short term, we note that die categories and conclusions were arrived at 
through much argumentation and compromise witii colleagues with first-hand knowledge of the data. 

The third shortcoming is that only design problem protocols have been examined. This only allow 
us to make the weak claim that we have identified certain invariants in the design problem space. It does 
not permit die additional claim diat Uiese invariants are not (collectively) found in nondesign problem 
spaces. This latter claim is desirable for Uie motivation of generic design as a useful theoretical construct. 
But it requires Uie examination of nondesign protocols. The comparison of nondesign problem spaces witii 
design problem spaces is a matter of ongoing concern. 

4J. Future Work 

This investigation has been a first-pass, breadUi-first look at design problem solving. We have tried 
to lay out the. major pieces of die design problem space and explain or justify them by an appeal to die 
design task environment and the stmcnire of die IPS. A logical extension of Uiis work would be to push 
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the analysis further and to derive a process model of design from it. 

In ccnccntrating on the big picture, we have had to resist the temptation to delve deeply into any sin* 
gle feature of the problem space. Of particular interest to us are the phenomena of scenario immersion, 
leaky modules, and the use of artificial symbol systems. Each of these promises to be a rich and intricate 
field of study. 

Also, we have not said anything about the differences in the problem spaces of our three subjects. 
We have notice4 some interesting differences in their knowledge bases, the external symbol systems they 
use, and their cultural and professional values and practices. However, any conclusions in this v^i&oid must 
wait until we gather and analyize additional data. 
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Fig. 7: First Rough Sketch of (Vertical) Section of Post Office. See text. 
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